Closing this in favor of just including the columns' dtypes in the PackedColumns object in #8153, and sorting out the associated serialization/deserialization in another PR.

I think the logic will end up being very similar to _copy_type_metadata, where it may still be worth investigating how to reuse some of it. Especially given that we now also have a _copy_type_metadata_from_arrow, we'll end up having three pieces of very similar looking code.

Agreed - I think that a general function that is able to apply metadata to a column taking a dtype as input could work for all these cases, although I'm not sure how to handling typing if we want it to work for Arrow arrays.

Would fleshing out the casting functions here be helpful for this purpose?

I think all three cases could be implemented in terms of something like _apply_type_metadata.

class ColumnBase:

    def _apply_type_metadata(self, dtype: Dtype) -> ColumnBase:
        # return a new column composed of the data from `self`
        # and metadata from `dtype`
        pass

    def _copy_type_metadata(self, other: ColumnBase) -> ColumnBase:
        # copy the type metadata from the column `other` onto `self`
        return self._apply_type_metadata(other.dtype)
        
    def _copy_type_metadata_from_arrow(self, other: pa.Array) -> ColumnBase:
        # copy the type metadata from the pa.Array `other` onto `self`
        return self._apply_type_metadata(cudf_dtype_from_pa_type(other.type))

I'm not necessarily advocating that we need three distinct methods though. I think just having the first one is enough.

The difficult thing of reusing shared recursive backbone is that both cudf and pyarrow have nested column structure, where each column has there own dtype/type attribute. _apply_type_metadata should apply dtype to the current level, and recursively calls the correct _copy_type_metadata* depending on the child type (whether a cudf column or arrow array).

Right, I imagine that _apply_type_metadata would need to be recursive. In the arrow case, cudf_dtype_from_pa_type(other.type) will recursively construct the appropriate cuDF dtype from the arrow type. Would that work, or maybe I've missed something?

I could be wrong. For concrete example, say we have a column of list of list of ints:
In cudf,

>>> x = cudf.Series([[[1, 2, 3]]])

It is true that x's dtype needs to be recursively constructed, so you have:

>>> x.dtype
ListDtype(ListDtype(int64))

However, x's child column (which is a separate Column instance in cudf), also has a dtype that requires to be recursively constructed:

>>> s._column.children[1].dtype
ListDtype(int64)

This is best illustrated if the nested type is not only a ListDtype(int64) column, but a more deeply nested (e.g. structs of lists). But for simplicity this should do the justice.

For pyarrow it's pretty much the same.

>>> x = pa.array([[[1, 2, 3]]])
>>> x.type
ListType(list<item: list<item: int64>>)
>>> x.values.type
ListType(list<item: int64>)